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Abstract Body 


Background / Context: 

Description of prior research and its intellectual context. 

In recent decades, much attention has been focused on student achievement in the United States. 
Many policy initiatives have been attempted in an effort to bolster achievement, including 
increasing school revenue, decreasing class size, expanding early childhood programs, and 
introducing vouchers and charter schools, to name a few, but not all of these initiatives have had 
the desired impact. Research has shown that other, less tangible factors such as teacher quality 
and the characteristics of a student’s peers may play a much greater role. This research will focus 
on one of these less tangible factors: principal quality 

Compared to the large, well-established teacher quality literature, there exists a relatively small 
quantitative literature on principals. 1 Recent evidence finds that more experienced principals 
improve school performance (Clark, Martorell, and Rockoff, 2009), principals that spend more 
time dedicated to organizational management lead schools that have higher state-assigned grades 
(Horng, Klasik, and Loeb, 2009), they import their policies and practices from one school to 
another (Cannon, Figlio, and Sass, 2012), self-assessment of principal organizational 
management skills predicts growth in state-assigned school grades (Grissom and Loeb, 
forthcoming), and principals are motivated by the opportunity to change schools (Cullen and 
Mazzeo, 2007). 

A small and more recent literature estimates principal value added to test score gains, using 
methods similar to those used to estimate teacher value added. Dhuey and Smith (2012) estimate 
fixed effects for principals in British Columbia, Canada, finding substantial variation across 
principals in terms of both math and reading value added. Grissom, Kalogrides and Loeb (2012) 
compare and contrast the results obtained from variations of principal value added models to 
each other and to non-test-based assessments of principal quality. They find that among the 
models they estimate, measuring principal quality with a principal by school fixed effect is most 
correlated with non-test-based measures. Branch, Hanushek, and Rivkin (2012) use a principal 
by school fixed effect approach and find significant variation in principal quality, which varies 
positively with the poverty level of the school. Using aggregate data on teachers, they find that 
teacher exits are related to principal quality. Finally, Coelli and Green (2012) estimate the lower 
bound of the variance of principal effects on graduation probabilities and grade 12 provincial 
final exam score in British Columbia, Canada. 

Purpose / Objective / Research Question / Focus of Study: 

Description of the focus of the research. 

This paper has two main goals. The first is to measure principal quality by estimating principal 
value added to student achievement. We estimate a set of principal fixed effects from test score 
gains with data on North Carolina students between third and eighth grade. The second goal is 


1 See Hanushek (2006) for a review of the teacher quality literature. See Hallinger and Heck (1998) for a review of 
the qualitative principal literature. 
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use the estimated principal value added to determine what makes a principal better than another 
at raising test score value added. 

Setting: 

Description of the research location. 

n/a - secondary analysis of already compiled data from North Carolina 

Population / Participants / Subjects: 

Description of the participants in the study: who , how many, key features, or characteristics. 

The primary data comes from all public schools in North Carolina from the 1998/99-2009/10 
school years. This data come from administrative records from the North Carolina Department of 
Public Instruction maintained and distributed by the North Carolina Education Research Data 
Center. Because we use value-added model in test scores to estimate principal quality, we restrict 
our focus to students that have a valid math or reading scores in year t and a valid math or 
reading score in year t-1. There are 5,407,020 student-years observed between grade 4 and grade 
8 between 1998/99 and 2009/10 school years that have valid test scores in both year t and t-1. 

We drop 9,724 observations from students who attend schools with less than 10 students. We 
also drop 491 observations from students who are too far ahead or behind in school for their age. 
Finally, we drop 8,262 observations that we cannot link to a particular school. Our final analysis 
sample consists of 5,388,543 student-year observations. 


Intervention / Program / Practice: 

Description of the intervention, program, or practice, including details of administration and duration. 

The intervention in this study is to test how much principals matter in the production of school 
achievement. We also examine the impact on various school inputs and outcomes of replacing 
the outgoing principal with an incoming principal who has different value added. 

Research Design: 

Description of the research design. 

To estimate the principal effects, we use the following value-added model of students’ test 
scores. 

yu = A +y*A + 4 A +fu)A + p P mA + +r h +£ u ( 1 ) 

where y u is the math or reading score for student i at time t; y tll is the student’s one year lagged 
math or reading score; x a is a vector of student-level demographic characteristics; z s(i t)t is a 
vector of school-level demographic characteristics for the school that student i attends at time /; 
Pp(U ) t ' s a vector °f principal-level demographic characteristics for student V s principal at time t; 

dpiuty ’ anc * Vt are time invariant principal, school, and year effects; s u h an idiosyncratic 
error term. 
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As a first step towards determining why some principals have higher value added than others, we 
relate the fixed effects estimated above to a small set of principal characteristics, including 
education, experience, and the state component of annual salary. We estimate the following 
specification by OLS: 

5 P = k +Ed p A +Ex P P k +Salary p fc+C p (2 ) 

The vector of education variables contain indicators for whether the principal has a bachelors, 
masters, doctorate, or advanced degree from a competitive or non-competitive institution. This 
regression is meant to be descriptive only, so we attach no causal interpretation to the 
coefficients. 


The second step of our analysis measures the effect on various school outcomes of a change in 
principals. We estimate specifications of the following form: 

O, = a 0 + <ir a \ +d!? l a 2 +dT* (X S + a \ + Ql-I a 5 + PQst-\ a 6 +Z x/ a 7+^+k + V tt ( 3 ) 

The independent variables of interest are a set of four dummies indicating the type of principal 
change a school experiences. The variable d™ se equals one when the school receives a new 
principal, and the new principal has higher value-added than the departing principal. Similarly, 


equals one when the incoming principal has lower value added than the outgoing principal, 
and equals one when the incoming principal and outgoing principal have similar value 

added. Finally, equals one when the incoming principal is not previously observed in the 
data. Differences in value added between the incoming and outgoing principal are assigned to 
rise, fall, or stay the same based on terciles of the difference in principal value added among all 
principal switches in year t. 


The school input and outcome measures we use are (1) percent AYP targets met (2) number of 
crimes per one hundred students (3) number of long term suspensions (4) percent of teachers 
with advanced degrees in t+1 (5) teacher turnover rate between t and t+1 (6) percent licensed 
teachers in school in t+1 (7) percent National Board Certified teachers in school in t+1 (8) 
percent daily attendance (8) percent of classes with highly qualified teachers (9) percent of 
teachers in school with 0-3 years of experience in t+1 (10) percent of teachers with 4-10 years of 
experience in t+1, and (11) percent of teachers with eleven or more years of experience in t+1. 


Statistical, Measurement, or Econometric Model: 

Description of the proposed new methods or novel applications of existing methods. 


We use a fixed effects approach to estimate principal value added. We use the above referenced 
models to determine why some principals have higher value added than others. 


Usefulness / Applicability of Method: 

Demonstration of the usefulness of the proposed methods using hypothetical or real data. 


Recently, policymakers have shown increased interest in evaluating school principals based on 
test score gains of students in their schools. Recent laws have been enacted in Florida, Louisiana, 
and Tennessee that utilize test score gains in evaluating principals. This research helps evaluate 
and understand how to provide an appropriate measurement of principals based on test score 
gains, if at all possible. 
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Data Collection and Analysis: 

Description of the methods for collecting and analyzing data. 

Data was obtained from the North Carolina Education Research Data Center. The data was 
analyzed using STATA. 

Findings / Results: 

Description of the main findings with specific details. 

We estimate that a one standard deviation of principal value added is approximately 0.13-0.18 in 
math and 0.10-0.14 in reading. Adjusting for sampling error shrinks the standard deviations by 
up to 15 percent. We also extend the model to include a fixed effect for each principal-school 
match. Depending on the model we use, the fixed principal standard deviations shrink to 0.04 in 
math and 0.02 in reading. The standard deviation of the match-specific component is 0.07 in 
math and 0.04 in reading. Though these estimates should be interpreted with caution due to the 
strong demands the models place on the data, they suggest that part of the principal component is 
actually attributable to the match between principal and school. 

We show that experience as a principal plays a small role in increasing value added in both math 
and reading, and there is some evidence that having an advanced or doctorate degree increases 
value added in reading. Second, we assess the impacts on various school outcomes of the arrival 
of a new principal at the school. More specifically, using principal fixed effects generated using 
the strategy from above, we estimate what happens when the incoming principal has higher, 
lower, or similar value added compared to the outgoing principal. We also assess the impact of 
replacing the outgoing principal with one that has no experience as a principal in North Carolina. 

We find that changing to a principal with higher value added lead to higher math and reading 
scores after a new principal arrives. Changing to a new principal leads to lower math and reading 
scores. In addition, replacing the current principal with one of higher value added increases the 
fraction of Adequate Yearly Progress (AYP) targets met, increases teacher turnover, and reduces 
slightly the number of national board certified teachers. When the incoming principal is not 
previously observed in the data, we see a negative effect on the percentage of AYP targets met, 
fewer suspensions, higher teacher turnover, and a higher fraction of teachers with low 
experience. Analyzing the effect of principals on such a variety of outcomes provides a broad 
analysis of the effect of principals. 

Conclusions: 

Description of conclusions, recommendations, and limitations based on findings. 

These results have important implications for policy. The main implication is that shifting 
principals between schools has the potential to significantly reduce achievement gaps. Policy 
makers can identify the most effective principals using available test score data, and allocate 
them between schools to potentially reduce achievement gaps. In addition, these results indicate 
that much more work needs to be completed to uncover what makes a good principal good. 
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